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Abstract 


Vaccination is the right method to deal with the increasing number of infections due to COVID-19. 
One method for assessing the effectiveness of a vaccine is by using a questionnaire. This pilot study 
was conducted to assess the validity and reliability of the Pfizer booster vaccine effectiveness 
questionnaire in Indonesia. The pilot study was carried out cross-sectionally using a new 
questionnaire that had not been tested for validity and reliability. The validity test of the 
questionnaire was carried out using the modified Delphi method which 3 professional will 
determine whether the questionnaire is valid to be used while in reliability analysis using inter 
reliability consistency analysis with score not less than < 0.3. Based on the collected data, there 
are 25 (62.5%) female respondents and 15 (37.5%) male respondents. With the age of majority 
<30 years as many as 37 (92.5%) respondents, <45 as many as 2 (5%) respondents and >45 as 
many as 1 (2.5%) person.. In the validity test, 3 experts stated that the questionnaire is considered 
to be valid and suitable for use in research. Before checking the inter reliability consistency there 
were 50 questions but after the analysis only 12 questions were considered reliable to be used with 
Cronbach's alpha score 0.797. Of the 12 questions, there were 3 questions on sociodemography, 3 
questions on side effects, 4 questions on allergy history, and 2 questions on data regarding history 


of COVID-19 infection. It can be concluded that the questions are considered valid by 3 experts 
and has 12 reliable questions in this questionnaire with Cronbach's alpha of 0.797. Although it 
functions as a pilot study for questionnaire development, its significance lies in the potential to 
enhance our understanding of the various factors that impact the effectivness of the pfizer booster 
vaccine, specifically within the Indonesian population. 


Keyword: Cronbach’s Alpha, Pfizer, Reliability, Vaccines, Validity 


INTRODUCTION 

To reduce the infection rate due to COVID-19, vaccination is the right method to 
overcome this problem by injecting to the body several doses of either 2, 3 or 4 doses given for 
a certain period of time such as several weeks or months an inactive or weakened part of an 
organism that is put into a liquid to form the immune system (Kemenkes RI, 2021; Speiser & 
Bachmann, 2020). Until 2023, there are many COVID-19 vaccines that are used by many 
nations and one of them is the Pfizer-BioNTech vaccine. Pfizer-BioNTech vaccine or Pfizer 
vaccine is one of is the most popular COVID-19 vaccine due to it’s 95% of efficacy which 
also can be proven through many research. In a research, there was a difference in the number 
of cases of infection due to COVID-19 where 39 cases of COVID-19 were found in the Pfizer 
vaccine group and 82 cases in the placebo group. Vaccine efficacy between the first and second 
doses was 52% (95% credible interval 29.5% to 68.4%). Vaccine efficacy then increased to 
95% (90.3% to 97.6%) seven or more days after the second dose proving the importance of 


vaccine (Mahase, 2020). Similar research also found to have the same result, where in vaccine 
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participants who had never been infected with COVID-19 or were currently infected with 
COVID-19 had an efficacy of 95% while in 9 participants who were infected with COVID-19 
after at least 7 days of giving the second dose the efficacy changed to 94.6% (Polack et al., 
2020). 

In current time as most first and second dose had already distributed to people all over 
the world, the use booster Pfizer becomes an option to gives more protection the body againts 
new variants of COVID-19. In a research Pfizer booster shown to have significant efficacy to 
protect it’s receiver from the COVID-19 infection by having 95.3% efficacy (Chi et al., 2022). 
This can be proven where in a specific research that study the efficacy of booster vaccine of 
the 21,707 recipients of booster types of mRNA-based COVID-19 vaccine, only 11.2% were 
infected with the omicron variant, 3.1% were infected with the delta variant and 85.6% were 
negative for SARS-CoV-2, different from the number that only received the vaccine until the 
second dose where 31,271 participants 23.2%, 14 .6% and 62.2% were infected with omicron, 
delta and SARS-CoV-2 negative variants (Accorsi et al., 2022). In other study, revealed that 
young individuals (40.7%), women (62.8%), Malays (63.8%), Muslims (72.3%), married 
individuals (52.9%), highly educated individuals (86.8%), and those in good health comprised 
the majority of respondents (85%) have concerns about the COVID-19 vaccine because of the 
side effects (95.8%), safety (84.7%), lack of knowledge (80.9%), effectiveness (63.6%), and 
cultural and religious considerations (20.8%) were the primary reasons for hesitancy. 

Safety is a critical criterion when assessing the use of a vaccine by monitoring it’s 
safety. The monitoring of system safety involves gathering information about medical issues 
and complications that occur following the administration of these pharmaceutical products, 
including both expected side effects and any unexpected adverse reactions (Medicines gency, 
2022). When a drug is administered to a significant population, there is always a risk of side 
effects. Therefore, it is crucial to closely monitor for any signs of direct or indirect 
consequences that may arise as a result (Ahamad et al., 2023). Out of the 370 participants 
surveyed, 20 individuals reported experiencing symptoms of nausea and vomiting (Al-Matouq 
et al., 2022). These results are consistent with a separate study that investigated the side effects 
of the Pfizer vaccine, which identified gastrointestinal symptoms in 10 cases of mild side 
effects and 1 case of severe side effects out of a sample of 46 individuals (Lee et al., 2022). 

Although Pfizer booster vaccine proves to be safe and effective, there are many people 
in Indonesia that stills too afraid or believe it is better to not get vaccinated. In a population 
study, it was found that out of 11,611 participants who filled out the survey, 201 and 209 


participants preferred not to vaccinate because of fear of needles and personal beliefs 
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(Hidayana et al., 2022). Doubts on the effectiveness of booster vaccine also can be affected 
also by news media. In Malaysia itself, there are 38.5% that very much agree news media could 
affect their decisions on getting vaccinated (Kyaw et al., 2022). 

There are many methods to prove the effectiveness of booster vaccine and of them is 
by using a questionnaire. Questionnaires are a data collection method where research 
participants are presented with a series of questions to answer. This approach involves 
engaging individuals from the target population to ensure that the questionnaire accurately 
represents their perspective. The goal is to create a questionnaire that is acceptable, 
comprehensive, and relevant to their specific circumstances. (Ricci et al., 2019). 

While the Pfizer booster vaccine demonstrates high efficacy, it is not a guarantee that 
individuals will be completely immune to COVID-19 infection or hold the belief in the benefits 
of receiving the Pfizer booster vaccine. Variations in sociodemographic factors and geographic 
locations in Indonesia may contribute to the differing incidence of side effects and Covid-19 
infections following the administration of the Pfizer booster vaccine and until now, there are 
still few studies that analyze the instrument of the research’s instrument especially instrument 
that analyze the effectiveness of the Pfizer booster vaccine in Indonesia using a questionnaire. 

Therefore this pilot study was conducted to assess the validity and reliability of the 
questionnaire used to assess the effectiveness of the Pfizer booster vaccine in Indonesia which 
will provide better insight on what variables and questions that can be used to asses the 
effectiviness of pfizer booster vaccine in indonesia and provide basic info of pfizer booster 


vaccine effectiveness on a small group of samples. 


METHODS 

A cross-sectional study model with a questionnaire as a tool was conducted for 2-3 
months to obtained the needed data of the first 40 respondents. The main method of collecting 
primary data in this study involved distributing questionnaires on various social media 
platforms such as Twitter, Facebook, WhatsApp, Line, and Instagram with the population and 
sample of this study was sorted out with convenience sampling so therefore the obtained 
population of this study is Indonesian citizen, while the samples in this study include people 
who have received the Pfizer booster vaccine on the third dose and aged >18 years. Participants 
who has autoimmune, cancer, HIV/AIDS or hepatitis and also pregnant women are excluded. 
Prior to completing the questionnaire, the researcher provided a clear explanation to the 
respondents regarding the research aims and objectives. But before the questionnaires were 


distributed an ethical approval was made. Based on ethical approval, this research was 
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approved by the Health Research Ethics Committee of the University of 17 August 1945 
Jakarta with reference number No.58/KEPK-UTA45JKT/EC/EXE/01/2023 and was declared 
ethically feasible according to 7 (seven) WHO Standards 2011. 

Data in this pilot study were divided into 4, namely sociodemographic data, vaccine 
side effects, history of allergies and data on the history of COVID-19 infection in respondents. 
The data that has been obtained will be sorted according to the predefined inclusions and 
exclusions. After sorting, the data from the questionnaires were analyzed for validity using a 
modified Delphi study and for reliability by asess the result of internal reliability consistency 
analysis In the final results of the analysis, the questionnaire was declared valid if declared 
appropriate by three experts through an expert judgment letter and declared reliable if Cronbach 


Alpha > 0.07 and Corrected item-total correlation >0.3. 


RESULTS 

In the validity test, the researcher makes an assessment table which will later be ticked 
by the expert who is the tester for the validity of the questionnaire. Based on the reviews 
provided, the questionnaire that has been made is stated to be representative, relevant, and clear 


or unbiased. 


Table 1 Characteristics of Respondents (n=40) 


1 Gender 
Male 15 (37.5%) 
Female 25 (62.5%) 
2 Age 
<30 37 (92.5%) 
<45 2 (5%) 
>45 1 (2.5%) 


In the pilot test, reliability and validity analysis was carried out on as many as 40 
respondents. There are 25 (62.5%) female respondents and 15 (37.5%) male respondents. With 
the age of majority <30 years as many as 37 (92.5%) respondents, <45 as many as 2 (5%) 


respondents and >45 as many as 1 (2.5%) person. 


Table 2. Internal reliability consistency test 
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Instrumen Total Scale Scale Corrected Cronbach’s 
item mean if varience  item-total alpha if item 

item if item correlation deleted 
deleted deleted 

Sosiodemography 

What is your gender? 19.17 22.282 0.355 0.790 

What was the first dose of vaccine you 3 18.23 14.668 0.671 0.768 

received? 

What second dose of vaccine did you 18.23 13.978 0.789 0.741 

receive? 

Side effects 

Do you feel headache / dizziness after 19.13 21.223 0.589 0.774 

receiving the booster vaccine? 

Do you feel chills after receiving the 3 19.40 21.697 0.589 0.778 

booster vaccine? 

Do you feel nauseous / vomit after 19.00 22.414 0.342 0.791 

receiving the booster vaccine? 

Allergy history 

After the booster vaccine, have you 18.90 21.955 0.494 0.783 

ever had an allergy? 

After the booster vaccine, do you feel 18.70 23.321 0.349 0.795 

itching after consuming peanuts? 4 

After the booster vaccine, do you feel 18.77 22.530 0.485 0.787 

itching after consuming fish? 

Ifyou had allergies before the vaccine, 18.90 22.369 0.392 0.788 

what allergic condition do you feel 

now? 

Data regarding history of COVID- 

19 infection 

Have you ever experienced fever 19.40 22.179 0.463 0.785 

symptoms after getting the second 2 

dose of vaccine? 

How long have you had fever 19.13 20.326 0.409 0.786 


symptoms? 


In testing internal reliability consistency which can be seen in table 2, it was found that 


from the 50 questions tested, there are only 12 questions that are considered valid based on the 


corrected item-total correlation score with a score of > 0.3. The reliable questions consist of; 3 


(three) sociodemographic instrument questions, 3 (three) side effect instrument questions, 4 


(four) allergy history questions and 2 (two) data instrument questions regarding the history of 


COVID-19 infection. 


Table 3. Reliability analysis 
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Cronbach's Alpha Cronbach's Alpha Based N of Items 
on Standardized Items 


0.797 0.834 12 


From the results of the reliability analysis of the questionnaire in table 1, there are 12 


reliable items with a Cronbach alpha value for 12 items of 0.797. 


DISCUSSION 

Pfizer is an mRNA-based vaccine that can enhance the immune response against the 
Covid-19 virus. The vaccine utilizes mRNA extracted from the SARS-CoV-2 virus, 
specifically targeting the spike protein found on its surface. This mRNA is encapsulated within 
lipid nanoparticles. Upon intramuscular injection, these nanoparticles attach to host cells and 
release the mRNA into the cytoplasm. Ribosomes within the cells utilize the mRNA to 
synthesize the viral spike protein. The resulting proteins, including MHC-2 found in antigen- 
presenting cells and MHC-1 present in all nucleated cells of the body, are then expressed on 
the cell membrane. This mechanism stimulates the immune system to recognize and mount a 
response against the Covid-19 virus (Mascellino et al., 2021). 

To asses the effectiveness of pfizer booster vaccine a pilot research is needed as it is 
the first step before the primary research to assess the clarity and comprehensibility of scale 
items and evaluate the scale's reliability through the calculation of internal consistency. 
Questionnaires are commonly used to accomodate data collection. In a research that investigate 
the COVID-19 the initial efficacy of the primary vaccine series, its gradual decline over time, 
and the safety and effectiveness of administering booster doses within a community in the 
United Kingdom they were able to find a conclusion of booster dose could restore the primary 
vaccine effectiveness notably the pfizer booster vaccine by using questionnaire as their tool of 
data collection (Menni et al., 2022; Yilmaz et al., 2018). 

There are mainly 2 setting of questions that can be used in a survey such as close ended 
and open ended question. The type of question that is used in this study is a combination of 
open ended and close ended question because to collect data from the selected population it is 
better to use an open ended or close ended questions or by combining both where open-ended 
questions empower respondents to express themselves in their own terms, fostering rich and 
diverse responses. Open ended questions are proveen to be beneficial when researchers are 
uncertain about how participants may answer, as well as for creating fresh response choices for 


closed-ended questions (Story & Tait, 2019). Conversely, close-ended questions restrict the 
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range of potential answers by offering predefined choices, potentially constraining the accuracy 
and depth of responses (Taherdoost, 2022). Combined method is also can be found in a similar 
study to obtain the participants opinion on post-vaccination experienced (Sultana et al., 2023). 
In contemporary research, validity and reliability are fundamental concepts that play a 
pivotal role in improving the exactness and dependability of evaluating and appraising a 
research endeavor. Validation is an integral component of quality management procedures, yet 
it remains less prevalent in pharmacy practice and research, necessitating education and 
training during its incorporation into pharmaceutical research. Reliability, on the other hand, 
denotes that a scale or test is dependable, yielding consistent results with repeated 
measurements under consistent conditions (Ahmed Alomi, 2020; Ahmed & Ishtiaq, 2021). 
The validation of questionnaire was done using a modified Delphi method by collecting 
the opinions of three clinical pharmacists. The data validation process is critical to ensure that 
the survey questionnaire is filled in accurately and represents reliable response data. The Delphi 
technique is a systematic and collaborative approach to forecasting, utilizing the combined 
opinions of a panel of experts. This structured method of consensus-building among panel 
members has been widely embraced across various medical disciplines. In recent decades, the 
Delphi methodology has played a crucial role in developing best practice guidelines by 
leveraging collective intelligence. It is particularly valuable in situations where research is 
scarce, ethical or logistical challenges exist, or when available evidence presents conflicting 
information (Nasa et al., 2021). Delphi panels are typically formed through a purposive or 
convenience sampling approach, which involves selecting experts based on specific 
qualifications (Plaiasu et al., 2023). In consensus methods, there are not strict guidelines 
regarding the inclusion of participants, except that each participant should be considered an 
expert in the relevant subject matter and be representative of their profession. Additionally, 
participants may be selected based on their ability to implement the findings or their status as 
unchallenged experts in the field (Taylor, 2020). Based on the validity test, the three panelists 
approved the questions from this questionnaire to be use to collect data of participants. 
Reliability test is a test used to prove whether the tests performed are reliable or not and 
to test the reliability of the questionnaire. One of the most common method to asses the 
reliability is by the result of the internal reliability consistency test. The internal reliability 
consistency test is a test that measures the degree of relatedness between the items included in 
the test. To assess the reliability of the scale, the most commonly used statistic is Cronbach's 
coefficient a. (Bornstein, 2018). Cronbach's alpha is a statistical measure used to evaluate 


internal consistency and reliability of a measurement scale. A Cronbach's alpha score above 
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0.7 indicates acceptable consistency, above 0.8 indicates good consistency, and above 0.9 
indicates excellent consistency (Wilson et al., 2022). From the result of this study, The 
reliability analysis of the questionnaire in table 1, there are 12 reliable items with a Cronbach 
alpha value for 12 items of 0.797. 

In this study, an issue arises in the Reliability analysis. Validity and reliability analyses 
are interconnected. While a question may be deemed valid according to expert opinion, it may 
lack reliability as per the reliability analysis. Same as validity, Even if a test or measurement 
demonstrates high reliability, indicating consistent results when applied repeatedly, it does not 
necessarily mean that the test accurately assesses the desired behavior or quality it intends to 
measure (Surucu & Maslakci, 2020). This issue becomes evident when out of 50 questions in 
this study, only 12 demonstrate reliable internal consistency exceeding 0.3. To tackle internal 
reliability concerns, researchers should ask themselves: Can another researcher readily 
replicate this study based on the provided description? How probable are similar results and 
analyses if the study were conducted anew? (Rose & Johnson, 2020). 

Additional investigation is required, involving another group of sample, in order to 
know whether are there more questions that are reliable from the 50 questions. This study have 
the potential to offer further understanding of the elements that play a role in the effectiveness 


pfizer booster vaccine, especially among Indonesians citizens. 


CONCLUSIONS 

Based on this pilot test, it has been determined that the survey questions validated by 
three experts are both valid and reliable, as indicated by a Cronbach's alpha coefficient of 0.797 
for the 12 questions, determined through an internal reliability consistency test. It's important 
to note that this research represents an initial investigation into the effectiveness of the Pfizer 
Booster vaccine in Indonesia. While it serves as a pilot study for questionnair, This study holds 
the potential to provide a deeper comprehension of the factors influencing the effectiveness of 


the Pfizer booster vaccine, particularly within the context of Indonesian citizens. 
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